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10990102-1 

HIGH RELIABILTY FORENSIC COLOR MARKING SYSTEM 

Field of the Invention 

5 The present invention relates generally to printing with color printing systems. It 

relates more particularly to a forensic marking system for color printers that accurately 
identifies the printer on which the output was printed, while minimizing the perceptibility 
of the forensic markings to the human eye, 

1 0 Background of the Invention 

Over the past several years, the quality of color images produced by computer 
printers such as inkjet printers and color laser printers has improved to the point where 
such images now rival those achievable through color photography. In addition, many 

15 such printers are capable of printing on a wide range of media, including paper stocks of 
various qualities, sizes and thicknesses. When an optical scanner is used in combination 
with a printer and the appropriate print media, original documents can be scanned and 
printed to produce copies which can be difficult to distinguish from the original. The 
scanner can be a separate peripheral device, or can be combined with the printer in a 

20 single "all in one" peripheral device, such as the OfficeJet™ Pro Series product family 
from Hewlett-Packard Company, the assignee of the present application. "All in one" 
devices, which may be connected to a computer or may be used as a stand-alone color 
copier without requiring an attached computer, commonly include an inkjet based 
printing system that has a removable printer cartridge. Others have laser printing based 

25 systems. An optical scanning system is also provided. A user can scan a document with 
the scanning system and then print the document with the printing system. Image 
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model number, serial number, or owner of the device, thus allowing the source of a 
particular printed copy to be traced to a particular system or person. However, since it is 
likely that most printed copies produced by the system will be of perfectly legitimate 
documents, it is highly desirable that the forensic information be virtually unnoticeable to 
5 the unaided human eye, in order to avoid undesirably degrading the print quality of the 
copy. As a result, the size, pattern, location, repetition, and color of the forensic 
markings must be carefully designed so as to ensure that the markings are imprinted in as 
visually unobtrusive a manner as possible. At the same time, the forensic markings must 
also be readily detectable and accurately interpretable so as to identify the device on 
10 which the copy was printed to be determined from the printed copy with near-perfect 
accuracy, should the genuineness of a document be called into question and thus its 
origin determined. 

Some previous passive marking systems print small forensic markings in blank 
areas of the image. However, when the print density of a page is sufficiently high, such 

15 as when a photographic image is printed, there may not be enough remaining blank space 
to accommodate sufficient forensic markings to ensure accurate recognition. Other 
previous passive marking systems print forensic markings in a color, frequently yellow, 
which is not readily perceived by the human eye. As a result, these systems do not 
necessarily require blank image space for the forensic markings, but rather they may 

20 print the yellow markings in regions of the media that also contain the image. However, 
since the image itself may already contain yellow image content, forensic markings 
overlaid in these regions may be corrupted to such a degree that they misidentify the 
copying device, unless a mechanism exists to validate the markings. 
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Consequently, there is still a need for a passive marking system which minimizes 
the impact of forensic markings on print quality by forensically marking the printed 
medium with a small amount of imperceptible information overlaid on the image content, 
while ensuring that the passive marking system that produced the printed medium can 
5 accurately be identified from the forensic markings. 

Summary of the Invention 

In a preferred embodiment, the present invention provides a forensic marking 
10 system that unobtrusively marks printed media with encoded system information which 
subsequently allows the system on which the media were printed to be identified from 
the media with a high degree of accuracy, even where the markings are superimposed on 
other image content on the media. Such a marking system serves as a deterrent to 
counterfeiting of valuable documents by allowing the counterfeited materials to be traced 
1 5 back to their source. 

The preferred forensic marking system has a processor which superimposes onto 
source data representing the image to be printed at least one forensic marking. The source 
data may be provided to the marking system in electronic form from a data interface 
coupled to the processor. In some embodiments, the forensic marking system may also 
20 include a scanner coupled to the processor for acquiring the electronic source data by 

optically scanning a source image contained on a printed medium. The forensic marking 
contains an information marker which identifies the system, and an checksum marker 
which validates the integrity of the information marker. The system also includes a 
printer coupled to the processor which receives the combined source data and forensic 
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markings, and prints each forensic marking on the resulting image using a predetermined 
color which has a low visual response to the human eye, but which is readily detectable 
by electronic scanning so that the system on which the marked image was printed can be 
accurately identified, even where a portion of the source data and a portion of a forensic 
5 marking are printed in the same image pixels. The marking system of the preferred 
embodiment prints images using cyan, magenta, and yellow colorant, with the forensic 
markings superimposed in yellow. The forensic marking may also include at least one 
location marker which locates the forensic marking on the printed medium. The 
rectangular forensic marking of the preferred embodiment uses two location markers, a 

10 first location marker located at one corner of the marking, and a second location marker 
different from the first location marker located at the diagonally opposite corner of the 
marking. In some embodiments, the forensic marking may additionally include at least 
one camouflage marker containing a randomized data pattern in order to reduce the 
perceptibility of the forensic marking where it is replicated in multiple locations on the 

15 printed medium. The perceptibility of the forensic markings may additionally be reduced 
by rotating some of the markings on the medium by a multiple of 90 degrees. In the 
preferred embodiment, the rectangular array of the forensic marking is a sixty- four pixel 
by sixty- four pixel matrix, which is organized into sixty-four eight pixel by eight pixel 
cells. Colorant for the forensic marking is deposited during printing in at most one pixel 

20 of each of the eight pixel by eight pixel cells. This results in a maximum print density of 
the forensic marking within the rectangular array of approximately 1.5 percent, which 
avoids any significantly degradation of the image quality. In the preferred embodiment, 
the forensic marking is replicated on the printed medium at 256 pixel-row and pixel- 
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column intervals. In addition, adjacent forensic markings are offset from each other by a 
one pixel-row. 

The present invention may also be implemented as a method for printing a 
forensically marked image with a printer. The forensic marking is printed using a 
5 predetermined color such as yellow which is difficult to detect by the human eye, but 
readily detectable by electronic scanning so as to identify the system. Such a method 
encodes system information identifying the printer into an information marker, calculates 
a checksum for the information marker, encodes the checksum into a checksum marker, 
superimposes at least one encoded forensic marking containing the information marker 

10 and the checksum marker onto source data so as to create forensically marked output 
data, and prints that output data on a print medium. In some embodiments, the method 
also adds to each forensic marking at least one location marker which identifies the 
location and rotation of the corresponding marking on the print medium. The forensic 
marking may also include at least one camouflage marker containing a randomized data 

15 pattern to reduce the visibility of the forensic markings replicated on the printed medium. 
In cases where the printer prints a swath of the image on the medium at a time, the 
invention may also embodied in a method which obtains a swath of image data for the 
source image, and if more than a threshold amount of color data exists in the swath of 
image data, superimposes onto the image data a swath of at least one forensic marking 

20 having an encoded pattern of image pixels in graphical form. Then the method prints the 
image data on the medium to produce a forensically marked image. This method may be 
repeated until all the swaths of image data are printed. Typically the method will be 
performed by the processor of the printing system. 
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Another embodiment of the present invention is a system for determining a 
device on which a printed medium was printed. The system has a media receptor which 
receives the printed medium, an image scanner in optical communication with the media 
handler for acquiring multicolor image data for the printed medium, a color separation 
5 unit coupled to the image scanner for segregating predetermined color image data from 
the multicolor image data, and a forensic identification unit coupled to the color 
separation unit for identifying forensic markings in the predetermined color image data 
placed on the printed medium by the device when the medium was printed. The forensic 
identification unit has a locator subsystem which locates individual forensic markings in 

10 the predetermined color image data, a validation subsystem which analyzes an individual 
forensic marking so to verify its integrity, and an identification subsystem which 
analyzes the individual forensic marking to identify the device on which the printed 
medium was printed. 

Yet another embodiment of the present invention is a method of determining the 

15 printing device on which a forensically marked printed image was printed. The printed 
image is color separated to isolate the image data for the predetermined color in which 
the marking was printed, and then the image data is scanned to locate a forensic marking. 
A checksum for the encoded information marker is calculated, and then compared with 
the printed checksum obtained by decoding the checksum marker. If the calculated 

20 checksum and the printed checksum match, then the validity of the particular forensic 
marking is assured, and the information marker is decoded to determine the printing 
device on which the image was printed. If the two checksum do not match, another 
forensic marking can be located and analyzed for validity. 
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In still another embodiment, the information marker of the forensic marking can 
alternatively or additionally indicate whether copying of the forensically marked image is 
allowable or prohibited. If the printing system determines that the source image contains 
a forensic marking with an information marker indicating that the source image should 
5 not be copied, the printer will be disabled and a printed copy will not be produced. 

Other aspects and advantages of the present invention will become apparent from 
the following detailed description, taken in conjunction with the accompanying drawings, 
illustrating by way of example the principles of the invention. 

1 0 Brief Description of the Drawings 

FIG. 1 is a schematic representation of a passive marking printing system 
embodying the present invention. 

FIG. 2A is an example source image associated with the passive marking printing 
15 system of FIG. 1. 

FIG. 2B is an example output image produced by the passive marking printing 
system of FIG. 1 for the example source image of FIG. 2 A. 

FIG. 3 is a schematic representation of a forensic marking produced by the 
printing system of FIG. 1. 
20 FIG. 4 is a schematic representation of the replication of forensic markings 

produced by the printing system of FIG. 1 on a printed output medium indicating the 
placement of multiple markings on the medium. 

FIG. 5 is an enlarged example forensic marking produced by the passive marking 
printing system of FIG. 1. 
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FIG. 6 is a schematic representation of the replication of forensic markings 
produced by the printing system of FIG. 1 on a printed output medium indicating an 
example orientation of the different markings on the medium. 

FIGS. 7A-7C are schematic representations of the colorant deposited in four-pixel 
5 portions of forensic markings added to an exemplary image printed by the system of FIG. 
1. 

FIG. 8 is a perspective view of an All-in-One scanner / printer / copier device 
incorporating the passive marking printing system of FIG. 1 

FIG. 9 is a perspective view of a color swath printer incorporating the passive 
10 marking printing system of FIG. 1 

FIG. 10 is a flowchart of a method for printing a forensically marked image on a 
print medium 16 using the color swath printer of FIG. 9. 

Description of the Preferred Embodiment 

15 

Referring now to the drawings, and more particularly to FIGS. 1, 2A, and 2B, 
there is illustrated a passive marking printing system 10 constructed in accordance with 
the present invention which adds to output images 14 printed on a medium 16 encoded 
forensic markings 20 capable of identifying the printing system 10. The system 10 
20 advantageously minimizes the impact of these forensic markings 20 on print quality by 
marking the printed medium 16 with a small amount of imperceptible information 
overlaid on the image content, while ensuring that the system 10 that produced the 
printed medium 16 can be identified with a high degree of accuracy from the forensic 
markings 20 on the printed medium 16. 
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A preferred embodiment of the printing system 10 includes a processor 30 for 
superimposing onto source data at least one forensic marking 20 having an encoded 
pattern of image pixels in graphical form so as to create output data corresponding to the 
forensically marked output image 14. The source data is preferentially provided to the 
5 processor 30 either in electronic form via a data interface 12, or from an optical scanner 
40 electrically connected to the processor 20 which acquires the source data from a 
source image 18. The construction and operation of optical scanners is well known to 
those skilled in the art, and is described in further detail in articles in the Hewlett- 
Packard Journal ["Designing a Scanner with Color Vision" (August 1993, p.52), and 

10 "Design of a 600-Pixel-per-Inch, 30-Bit Color Scanner" (February 1997, p. 54)], which 
are hereby incorporated by reference in their entirety. The preferred embodiment also 
includes a printer 50 electrically coupled to the processor 30 for printing the output data 
so as to produce the forensically marked image 14. As will be discussed subsequently in 
further detail, the forensic markings 20 are printed at certain locations, intervals, and 

15 orientations on the forensically marked image 14 using a color to which the human eye 
has a low visual response, but one which is readily detectable by electronic optical 
scanning. A preferred embodiment of the printer 50 is a color inkjet printer which 
includes a print mechanism 52 which uses cyan, magenta, yellow, and black colorant 
print cartridges 54 in order to print color images 14. Typically a color inkjet printer 50 

20 prints a horizontal portion, or swath, of an image 14 on the print medium 16 as the print 
cartridges 54 are scanned across the surface of the medium 16 in a scan direction 2. The 
construction and operation of color inkjet printers is well known to those skilled in the 
art, and is described in further detail in various articles in several editions of the Hewlett- 
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Packard Journal [Vol. 36, No. 5 (May 1985), Vol. 39, No. 4 (August 1988), Vol. 39, No. 
5 (October 1988), Vol. 43, No. 4 (August 1992), Vol 43, No. 6 (December 1992) and 
Vol. 45, No.l (February 1994)], which are hereby incorporated by reference in their 
entirety. In the preferred embodiment, yellow colorant is used for printing all of the 
5 forensic markings 20, because the unaided human eye is less sensitive to the color 
yellow, particularly when printed on a white or light-colored background. Because 
optical scanning performs color separation, as will also be discussed subsequently in 
further detail, portions of the forensic markings 20 may be superimposed on the output 
image 14 even in regions which also contain portions of the source image. 

10 Considering now the layout of the forensic markings 20 in further detail and with 

reference to FIGS. 3, 4, and 5, each forensic marking 20 of the preferred embodiment is 
organized as an eight row by eight column matrix of sixty-four cells, such as a cell 28. 
Each cell 28, in turn, is preferably organized as an eight row by eight column matrix of 
sixty-four pixels, such as a pixel 62. Therefore, each marking 20 is also a sixty-four 

15 pixel-row by sixty- four pixel-column matrix. Each pixel 62 is a programmatically 

selectable printable location on the medium 16. In addition to the use of yellow colorant, 
the perceptibility of the marking 20 to the human eye is further reduced by printing each 
marker cell 28 as a sparse matrix, with colorant being superimposed in at most one of the 
sixty-four pixels in each cell 28. This pixel is preferentially located at the same position 

20 within each cell - for example, the upper left corner - to provide uniform spacing 

between marked pixels. Therefore, the maximum print density of the forensic marking 
within the rectangular array is approximately 1.5 percent. In the preferred embodiment 
of a color inkjet printer, two drops of yellow ink are used as the colorant for a pixel. 
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While the single drop per cell embodiment just described is preferred, the present 
invention also contemplates depositing additional drops per cell, in order to allow more 
complex encoding patterns at the expense of increased print density. 

Considering now the content of the forensic markings 20 in further detail, each 
5 forensic marking 20 contains a plurality of markers. Each forensic marking 20 includes 
an information marker 22 indicative of the system 10, and a checksum marker 24 
indicative of the information marker 22. Each forensic marking 20 also includes at least 
one location marker 26 which locates and orients the marking 20 on the printed medium 
16 as will be discussed in further detail subsequently. Each marker 22,24,26 spans one or 

10 more cells 28. The cells .28 of a marker may all be contiguous, or a marker may be split 
into two or more sections of cells 28. Each marker 22,24,26 is printed on the medium 16 
in graphical form as an encoded cell pattern. 

In the preferred embodiment, the information marker 22 contains sufficient 
information to identify the system 10 which printed the output medium with a high 

15 degree of certainty. Identification information can include any of the manufacturer, 
model number, serial number, and purchaser or owner of the printing system 10. This 
information is encoded in the marker 22 and printed using a particular binary code. The 
information marker 22 of the preferred embodiment includes a machine type identifier 
section 22a to encode the type of system 10, and a machine serial number identifier 

20 section 22b to encode the serial number of the system 10. High accuracy identification 
of the system 10 is important in cases where a prohibited source image such as currency 
was illegally copied, in order not to misidentify an alleged counterfeiter. The checksum 
marker 24 contributes to high accuracy identification by providing redundancy to ensure 
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that all information-bearing pixels of the marking were printed correctly. In the preferred 
embodiment, the checksum marker 24 encodes the total number of cells used to encode 
the information marker 22. For example, in the exemplary forensic marking 20 of FIG. 
5, twelve cells 28 are printed out of the thirty-two total cells in the serial number section 
5 22b of the information marker. Therefore, a corresponding binary value of twelve is 
encoded in the checksum marker 24 (where cell 25a represents the least significant bit of 
the checksum, and cell 25b represents the most significant bit of the checksum). When a 
printed forensic marking 20 is analyzed to identify the system 10, the checksum marker 
24 is compared to the decoded information marker 22. If the two do not match, the 

10 present marking 20 will be ignored and another marking decoded instead. 

In an alternate embodiment of the present invention which has characteristics of 
an active counterfeit prevention system, the information marker 22 may contain a "copy 
prohibit" control flag alternatively or in addition to the identification information. The 
operation of a copier upon detection of this control flag will be discussed subsequently. 

15 Considering now the at least one location marker 26 in further detail, the marker 

26 contains a notable pattern which allows the location of the corresponding forensic 
marking 20 to be precisely determined. In the preferred embodiment, a location marker 
26 encompasses a two by two cell matrix. In addition, each forensic marking 20 of the 
preferred embodiment has two location markers 26, a start location marker 26a located at 

20 one corner periphery of the forensic marking, and a stop location marker 26b having a 
different encoded pattern from the start location marker 26a located at the diagonally 
opposite corner periphery of the forensic marking 20. The stop marker 26b contains four 
cells with identical patterns, while the start marker 26a has three cells with identical 
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patterns and one blank cell. By positioning two different location markers at opposite 
corners, and by ensuring that the four cells in each of the other two corners of the 
forensic marking 20 do not contain either the start marker 26a or stop marker 26b 
patterns, the forensic marking 20 can be reliably identified by electronic scanning 
5 regardless of the orientation of the source image 18 on the media receptor 42 of the 
scanner 40. 

Considering now the placement of the forensic markings 20 on the forensically 
marked image 14, and as best understood with reference to the exemplary illustration of 
FIG. 4, in the preferred embodiment the markings 20 (such as markings 20a, 20b, and 

10 20c), are replicated at certain locations, intervals, and orientations so as to provide a 

plurality both to provide redundancy so as to improve the accuracy of printing system 10 
determination in the event some of the markings 20 are corrupted, as well as to ensure 
that the markings 20 cannot easily be cut out of a printed medium 16 before making a 
subsequent copy of it so as to defeat the forensic marking scheme. As will be discussed 

15 subsequently in greater detail, superimposing portions of the forensic markings 20 in 

regions of the output image 14 which also contain portions of the source image 18 further 
reduces the ability to cut out forensic markings 20 without also cutting out a 
corresponding portion of the source image 18. The replicated markings 20a,20b,20c are 
located at predetermined pixel-row intervals 62 and pixel-column intervals 64 relative to 

20 each other. In the preferred embodiment and as depicted in FIG. 4, the pixel-column 

interval 64 is 256 pixels, which is equivalent to 32 cell-column intervals. The pixel-row 
interval 62 is 256 pixels (32 cell-rows) minimum; in the preferred embodiment any 
individual pixel-row interval 62 may be increased as required to avoid splitting forensic 
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markings 20 between adjacent horizontal swaths. Horizontally adjacent forensic 
markings are offset from each other by a predetermined number of pixel-rows. The 
preferred embodiment uses a pixel-row offset 66 of one. In addition, vertically adjacent 
forensic markings are offset from each other by a number of pixel-columns. In the 
5 preferred embodiment, the pixel-column offset 67 is randomized for different horizontal 
swaths of the printed medium 16, so as to minimize patterning of the forensic markings 
20 that might make the markings 20 more perceptible to the human eye. Other choices of 
pixel-row and pixel-column intervals and offsets, both predetermined or randomized, are 
possible using the present invention. 

10 Alternate embodiments employ additional mechanisms to further reduce the 

perceptibility of a pattern of identical replicated forensic markings 20a-h by the unaided 
human eye. As best understood with reference to FIG. 6, some of the forensic markings 
20 may be printed in an orientation which is rotated from certain others of the forensic 
markings by a multiple of substantially ninety degrees. In the exemplary illustration of 

15 FIG. 6, each forensic marking 20a-h is rotated by ninety degrees from the adjacent 
marking 20a-h in both the scan direction 2 and the media advance direction 4. Other 
rotation placements, such as randomly selecting a multiple of ninety degree rotation for 
each adjacent marking 20a-h, can alternatively be used. The forensic marking 20 also 
contains at least one camouflage marker 29 which can be used to further reduce the 

20 perceptibility of a pattern of identical replicated forensic markings 20a-h by the unaided 
human eye. Each camouflage marker 29 has one or more cells which do not contain 
either location, checksum, or system information. While in the preferred embodiment 
these cells are left blank so as to minimize print density of the marking 20, in alternate 
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embodiments they may be printed with random data so as to make at least some of the 
forensic markings 20a-h different from others. In addition, while the exemplary 
illustrations of FIGS. 3 and 6 show a forensic marking 20 having two camouflage 
markers 29 located as shown, different numbers and locations of camouflage markers 29 
5 within the forensic marking 20 is contemplated by the present invention. 

Considering now the superimposing of the forensic marking 20 on the source 
image 18 in further detail, and with reference to the exemplary marking patterns of FIGS. 
7A, 7B, and 7C, there are illustrated three examples of a four-pixel portion 70a-c of a 
forensic marking cell 28. The upper left pixel 62a of each cell portion 70a-c denotes the 

10 pixel of the cell 28 into which yellow colorant can be deposited to form the forensic 
marking 20, while the remaining pixels 62b-d will not receive any colorant for the 
forensic marking 20. All four pixels of cell portions 70a-c may be imprinted using the 
appropriate combination of cyan, magenta, yellow, and black colorants as required to 
render the source image 18. Cell portion 70a illustrates the case where the source image 

15 is blank in this portion. In this case, if the cell 28 is to be forensically marked, two drops 
of yellow colorant (indicated by the two rectangular markings) are deposited into pixel 
62a. Cell portion 70b illustrates the case where the source image contains cyan and 
magenta data in this portion. In this case, if the cell 28 is to be forensically marked, two 
drops of yellow colorant (indicated by the two rectangular markings) are deposited into 

20 pixel 62a along with the magenta and cyan drops (indicated by circular markings). Cell 
portion 70c illustrates the case where the source image contains yellow data in this 
portion. In this situation, no yellow forensic marking colorant will be added to that 
which will be deposited for the source image. 
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Because optical scanning can electronically separate yellow markings from cyan 
and magenta markings as will be discussed subsequently, forensic markings are 
detectable even where the forensic marking 20 is superimposed on a region of the source 
image containing cyan or magenta colorant. However, if the marking 20 is superimposed 
5 on a region containing yellow (or black, which contains yellow) colorant, the presence of 
extraneous yellow data in the scanned image will corrupt the forensic marking 20. If the 
location marker 26 is corrupted, the forensic marking may not be located at all. If the 
location marker 26 is valid but either or both of the information marker 22 and the 
checksum 24 are corrupted, then it is virtually certain that the checksum marker will not 

10 match the decoded information marker 22, and thus the corrupt nature of the marking 
will be detected so as to avoid misidentifying the printing system 10. 

Considering now alternative packaging configurations for the printing system 10 
of the present invention, and with reference to FIG. 8, the invention may be embodied in 
an "All-in-One" device 6 which incorporates the scanner 40, processor 30, and printer 50 

15 so as to function as a stand-alone copier which may be operated from a control panel 44. 
The device 6 may also include a data interface 12 adapted to receive in electronic form 
source data corresponding to the source medium 17 containing the source image 18 to be 
copied. The scanner 40 may take the form of a flatbed scanner (illustrated) in which the 
source image 18 is placed in a fixed position on the media receptor 42 and the optical 

20 scanning elements are moved with respect to the medium 17. Alternatively, the scanner 
40 may take the form of a sheet-feed scanner (not shown), in which the optical scanning 
elements are fixed and the medium 17 is moved with respect to the scanning elements. 
Alternatively, as best understood with reference to FIG. 9, the invention may be 
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embodied in a printer 8 which does not contain a scanner 40 but which does incorporate 
the processor 30, printer 50, and data interface 12. In operation, the printer 8 receives in 
electronic form the source image 18 to be printed from a device, typically a computer, 
connected to the data interface 12. In yet another alternative embodiment of the printing 
5 system 10, both the scanner 40 and the printer 50 are separately packaged and connected 
to a computer (not shown) having the processor 30. While the just-described 
configurations are illustrative of the breadth of the present invention, they are not 
intended to limit its scope to just these configurations. 

The present invention may also be implemented as a novel method for printing a 

10 forensically marked image on a print medium 16 using a color swath printer. The 

forensic markings 20 added to the source image 18 to be printed consists of a graphical 
pixel pattern formed by an arrangement of markers, each of which markers has a 
graphical pixel pattern. As best understood with reference to FIG. 10, the method begins 
with encoding SI 02 printing system information sufficient to identify the system 10 on 

15 which the print medium 16 was printed, described previously herein, into an information 
marker 22. Next, a checksum for the information marker 22 is calculated SI 04. The 
checksum preferably is the total number of cells 28 printed to encode the system 
information. For example, the information marker 22 of the forensic marking 20 of FIG. 
3 has 32 possible cells that could be printed to represent the system information, so the 

20 checksum would have a value in the range of zero to thirty- two. According to the 
method, this checksum value is then itself encoded SI 06 into a checksum marker 24. 
Start 26a and stop 26b location markers which serve to delimit the area encompassed by 
forensic markings 20 are defined SI 08; each of these markers has a different distinctive 
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graphical pixel pattern so that it can readily be identified. The graphical pixel pattern for 
the forensic marking 20 is then formed SI 10 with the start 26a and stop 26b markers 
bounding an area that includes the information marker 22, checksum marker 24, and a 
camouflage marker 29 whose contents can in some embodiments be determined 
5 dynamically at the time of printing so as to reduce the perceivability of the forensic 
markings 20 by the human eye due to replication on the print medium 16. After the 
forensic marking pattern is formed, at SI 12 the method next gets a swath of source data 
corresponding to the source image 18. Whether or not the forensic marking 20 gets 
added to the source data for printing depends on whether color data exists in the source 

10 data for the swath. In the preferred embodiment, if there is no color data in the swath, the 
swath will be printed using only black colorant in order to improve printing throughput 
(since printing black data typically requires fewer scans to fully print a swath), and thus 
taking the "No" branch of SI 14 will omit superimposing the forensic marking 20 onto 
the source data for the swath. Similarly, if the swath contains less than sixty- four pixels 

15 of data in the scan direction 2, taking the "No" branch of SI 16 will similarly omit 

superimposing the forensic marking 20 onto the source data for the swath, so as not to 
print only a partial forensic marking 20 in that swath. However, if the swath contains 
color data ("Yes" branch of SI 14) and that color data is more than sixty-four pixels in the 
scan direction 2 ("Yes" branch of SI 16), then at least one forensic marking 20 will be 

20 superimposed S122, preferably using yellow colorant, onto the source data for the swath 
to form an output data swath for printing SI 24 on the print medium. If there is more 
source data remaining to be printed as other swaths ("Yes" branch of SI 26), the method 
will get SI 12 the next swath of source data. If there is no more source data remaining to 
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be printed ("No" branch of SI 26), the method ends. In some embodiments, the forensic 
markings 20 for certain swaths may be further modified before printing. At S 1 1 8 ? 
randomized data may be inserted into the camouflage marker 29 to reduce the 
perceivability of the markings 20. Alternatively or in addition, each of the forensic 
5 markings 20 in the swath may be rotated SI 22 by a multiple of ninety degrees, which can 
also reduce the perceivability of the markings 20. 

The processor 30, as best understood with reference to FIG. 1, includes modules 
which may be implemented in hardware, firmware, software, or any combination thereof 
to perform the above-described method. A forensic marking unit 31 receives the system 

10 information 32 and source data, and performs the calculation, encoding, randomization, 
rotation, and superimposition required to form the forensic markings 20 in the output 
data sent to the printer 50. 

Other aspects of the present invention involve examination of the forensic 
markings 20 printed on a particular source medium 17. As best understood with 

15 reference to FIG. 1, the present invention may be implemented as a system 10 for 

identifying the printing device on which a particular medium 17 was printed via forensic 
markings 20 imprinted on the medium 17. The medium 17 is placed on the media 
receptor 42 and scanned by the scanner 40 to acquire multicolor electronic source data 
corresponding to the source image 18. The source data usually represents the primary 

20 colors red, green, and blue. The processor 30 processes the source data. The processor 
30 has a color separation unit 33 coupled to the image scanner 40 for transforming and 
separating the primary color image data into data planes for each of the secondary colors 
cyan, magenta, and yellow. The processor also has a forensic identification unit 34 
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coupled to the color separation unit 33 and which receives the yellow color plane image 
data from the separation unit 33. The forensic identification unit 34 has a locator module 
35 for scanning the yellow color plane data to locate data representative of a forensic 
marking 20 therein. The forensic identification unit 34 also has a validation subsystem 
for analyzing an individual forensic marking 20 by calculating a checksum for the 
information marker 22 of the marking 20, decoding the checksum marker 24 to 
determine the imprinted checksum, and comparing the calculated checksum versus the 
imprinted checksum; if the checksums match, the analyzed forensic marking 20 is valid. 
The forensic identification unit 34 further has an identification module 36 for decoding 
the information marker of a validated forensic marking 20 to determine the printing 
device on which the source medium 17 was printed. 

Another embodiment of the present invention related to examination of the 
forensic markings printed on a particular source medium 17 is a printing system which 
modifies its operation if copy-prohibited subject matter is detected. As heretofore 
described, the information marker 22 may contain a "copy prohibit" control flag. In this 
embodiment, the forensic identification unit 34 analyzes the forensic markings 20 during 
copying of the document, and if the decoded information marker 22 indicates that the 
source image 18 should not be copied, the system 10 will not produce a copy. This 
capability is very useful in conjunction with controlled-copy documents for which 
unauthorized duplication is prohibited. Superimposing the information marker 22 on the 
original documents when they are printed will prevent them from being subsequently 
copied on the system 10. 



21 



10990102-1 

From the foregoing it will be appreciated that the printing systems and methods 
provided by the present invention represent a significant advance in the art. A printer can 
be constructed according to the present invention so as to provide a forensic marking 
system that unobtrusively marks printed media with encoded system information. The 
5 encoded system information can subsequently identify the system on which the media 
were printed with a high degree of accuracy, even where the markings are superimposed 
on other image content. Although several specific embodiments of the invention have 
been described and illustrated, the invention is not to be limited to the specific methods, 
forms, or arrangements of parts so described and illustrated. In particular, the invention 

1 0 may be used with single pass swath printers where a swath is fully printed during a single 
pass of the printheads 54 over the print medium 16, or with multipass swath printers 
where multiple passes of the printheads 54 over the print medium 16 are required to fully 
print a swath. It can be used with all types of swath printers including band printers and 
drum printers. It can be practiced with all types of inkjet printers including those which 

15 use thermal and piezo printing technologies; and with non-swath or non-inkjet color 
printers such as color laser printers. The invention is limited only by the claims. 
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What is claimed is: 



1. A system for printing a forensically marked image, comprising: 
a processor for superimposing onto source data at least one forensic marking so as 
5 to create output data corresponding to the forensically marked image, the at least one 
forensic marking having 

an information marker indicative of the system, and 
a checksum marker proximate the information marker and indicative of 
the information marker; and 
10 a printer coupled to the processor for printing the output data so as to produce the 

forensically marked image, the at least one forensic marking printed in a predetermined 
color having a low visual response to the human eye, but readily detectable by electronic 
scanning so as to identify the system. 



15 2, The system of claim 1, wherein the forensic marker includes: 

at least one location marker indicative of a location of the forensic marking 
proximate the information marker and the checksum marker. 



3. The system of claim 1, wherein a portion of the source data and a portion of the 
20 at least one forensic marking are printed in the same image pixels on a printed medium. 

4. The system of claim 1, wherein the information marker identifies the system 
sufficiently to enable tracing of the forensically marked image to the system. 
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5. The system of claim 1, wherein the information marker indicates that copying 
of the forensically marked image is prohibited, 

5 6. The system of claim 1, further including: 

a scanner coupled to the processor for acquiring the source data from a source 

image. 

7. The system of claim 1, further including: 

10 a data interface coupled to the processor for receiving in electronic form the 

source data corresponding to a source image. 

8. The system of claim 1, wherein the forensic marking further includes: 

at least one camouflage marker indicative of randomized data such that the 
15 visibility of the forensic markings due to replication on the forensically marked image is 
reduced. 

9. The system of claim 2, wherein the forensic marking is rectangular the 
forensic marking having 

20 a first location marker located at one corner of the forensic marking, and 

a second location marker different from the first location marker located at the 
diagonally opposite corner of the forensic marking. 
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10. The system of claim 9, wherein certain ones of the at least one forensic 
marking are rotated from certain others of the at least one forensic marking by a multiple 
of substantially 90 degrees, so as to reduce visual perceptibility of the pattern. 

5 11. The system of claim 1, wherein the printer prints color images using cyan, 

magenta, and yellow colorant. 

12. The system of claim 11, wherein the predetermined color is yellow. 

10 13. The system of claim 12, wherein the forensic marking is an encoded pattern of 

yellow image pixels that is superimposed on source data which is printed with at least 
one of cyan, magenta, and yellow inks. 

14. The system of claim 1, wherein each individual one of the at least one 
1 5 forensic marking is a rectangular array of image pixels. 

15. The system of claim 14, wherein the rectangular array is a sixty-four pixel by 
sixty-four pixel matrix. 

20 16. The system of claim 15, wherein the rectangular array is divided into sixty- 

four eight pixel by eight pixel cells. 
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17. The system of claim 16, wherein colorant is deposited during printing in a 
predetermined location in at most one pixel of each of the eight pixel by eight pixel cells, 
such that the image quality is not significantly degraded by the forensic marking. 

5 18. The system of claim 17, wherein a maximum print density of the forensic 

marking within the rectangular array is less than five percent. 

19. The system of claim 17, wherein a maximum print density of the forensic 
marking within the rectangular array is approximately 1.5 percent. 

10 

20. The system of claim 1, wherein an individual forensic marking is replicated in 
multiple locations on the forensically marked image. 

21. The system of claim 20, wherein the multiple locations are at predetermined 
1 5 pixel-row and pixel-column intervals 

22. The system of claim 21, wherein the pixel-row and pixel-column intervals are 
256 pixels. 

20 23. The system of claim 22, wherein adjacent forensic markings are offset from 

each other by a predetermined number of pixel-rows. 
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24. The system of claim 23 , wherein the predetermined number of pixel-rows is 

one. 



25. The system of claim 6, further comprising: 
5 detection means coupled to the scanner for detecting whether the source image 

contains copy-prohibited subject matter, and 

control means responsive to the detection means and coupled to the printer for 
disabling the copying of the copy-prohibited subject matter. 



10 26. A system for determining a device on which a printed medium was printed, 

comprising: 

a media receptor adapted to receive the printed medium; 

an image scanner in optical communication with the media handler for acquiring 
multicolor image data for the printed medium; 
1 5 a color separation unit coupled to the image scanner for segregating 

predetermined color image data from the multicolor image data; and 

a forensic identification unit coupled to the color separation unit for identifying in 
the predetermined color image data at least one forensic marking placed on the printed 
medium by the device, the forensic identification unit having 
20 a locator subsystem for locating an individual one of the at least one 

forensic marking within the predetermined color image data, 

a validation subsystem for analyzing the individual forensic marking so as 
to verify its integrity, and 
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an identification subsystem for analyzing the individual forensic marking 
so as to identify the device on which the printed medium was printed. 

27. A method for printing a forensically marked image with a printer, comprising: 
5 encoding system information identifying the printer into an information marker; 

calculating a checksum for the information marker; 
encoding the checksum into a checksum marker; 

superimposing onto source data at least one encoded forensic marking containing 
the information marker and the checksum marker so as to create output data 
10 corresponding to the forensically marked image; and 

printing the output data on a print medium so as to produce the forensically 
marked image, the at least one forensic marking printed using a predetermined color 
difficult to detect by the human eye, but readily detectable by electronic scanning so as to 
identify the system. 

15 

28. The method of claim 27, further comprising: 

adding to each of the at least one forensic markings at least one location marker 
indicative of a location of the forensic marking on the print medium. 

20 29. The method of claim 28, wherein the at least one location marker is further 

indicative of the orientation of the forensic marking on the print medium. 

30. The method of claim 27, wherein the printing the output data further includes 
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printing the at least one forensic marking using yellow colorant. 



31. The method of claim 27, farther including; 

rotating at least some of the individual ones of the at least one forensic marking 
5 by a multiple of 90 degrees prior to the printing 



32. The method of claim 27, further including: 

adding to each of the at least one forensic markings at least one camouflage 
marker indicative of randomized data such that the visibility of the forensic markings due 
10 to replication on the print medium is reduced. 



33. A method for printing a forensically marked image on a print medium with a 
swath printer, comprising: 

obtaining a swath of image data corresponding to a source image; 
15 if more than a threshold amount of color data exists in the swath of image data, 

superimposing onto the image data a swath of at least one forensic marking having an 
encoded pattern of image pixels in graphical form; 

printing the image data on the print medium so as to generate a swath of the 
forensically marked image; and 
20 repeating the obtaining, superimposing, and printing until ail the swaths of image 

data are printed. 
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34. The method of claim 33, wherein the threshold amount of color data is 64 

pixels. 

35. A method of determining the printing device on which a forensically marked 
5 image was printed, the forensically marked image having at least one forensic marking 

including an information marker indicative of the printing device and a checksum marker 
indicative of the information marker, comprising: 

scanning the forensically marked image to locate an individual one of the at least 
one forensic marking; 

10 calculating a calculated checksum for the information marker of the individual 

forensic marking; 

decoding the checksum marker of the individual forensic marking to determine a 
printed checksum; and 

if the calculated checksum matches the printed checksum, decoding the 
15 information marker to determine the printing device. 

36. The method of claim 35, wherein the at least one forensic marking is of a 
predetermined color, and wherein the scanning includes: 

color separating the scanned image to isolate image information for the 
20 predetermined color. 

37. A printer, comprising: 

an interface adapted for receiving printable image data; 
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a processor for 

encoding system information identifying the printer into an information 

marker, 

calculating a checksum for the information marker, 
5 encoding the checksum into a checksum marker, 

combining the information marker and the checksum marker into a 
forensic marking, and 

superimposing onto the printable image data at least one copy of the 
forensic marking; and 

10 a print mechanism coupled to the processor for printing the printable image data 

in a graphic form detectable by electronic scanning but undetectable to the human eye. 
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Abstract 

A forensic marking system and method for identifying the printing device on 
which a medium was printed. Encoded forensic markings which identify the device but 
are of low perceptibility to the human eye are superimposed on the printed image. The 
markings have a color, size, density, and repetition pattern that only minimally affect 
print quality while providing highly accurate identification of the printing device, even if 
the printed medium contains high-density image content. 



32 








to 


£ 
















s 

c 

E 


ing cells 






>er encod 




led pal 
ifier 




repeal 
ident 


£ 





</5 

c 

o 

•s 

is 



x 



•n 

U - X 

il SU J 



£ * 
5* 



B It 

"3 



Shu 



ii 



SB 




i 

3- 




vJ3 




a H 



f START ; 



Encode printing system information 
into an information marker 



6rc>4 



Calculate a checksum for the information marker 



[ 



Encode the checksum into a checksum marker 



T 



I Define start and stop location markers 
i \ having different distinctive patterns 




Form a forensic marking bounded by the start 
and stop location markers, and containing the 
! ^information marker, the checksum marker, and a 
camouflage marker 



Get a swath of source data for a source image 



•sua, 



~7 



Randomize the camouflage marker data 
for the forensic markings in the swath 



i 



Rotate the forensic markings in the 
swath by a multiple of 90 degrees 



i 



Superimpose the forensic marking in yellow 
onto the source data swath 
to form an output data swath 



Print the output data swath 
on the print medium 




PATENT APPLICATION 



DECLARATION AND POWER OF ATTORNEY 
FOR PATENT APPLICATION 



ATTORNEY DOCKET NO. 1Q99Q1Q2-1 



As a below named inventor, I hereby declare that: 

My residence/post office address and citizenship are as stated below next to my name; 

I believe I am the original, first and sole inventor (if only one name is listed below) or an original, first and 
joint inventor (if plural names are listed below) of the subject matter which is claimed and for which a 
patent is sought on the invention entitled: 

High Reliablity Forensic Color Marking System 



the specification of which is attached hereto unless the following box is checked 

( ) was filed on _ 
Number 



as US Application Serial No. or PCT International Application 

and was amended on : (if applicable). 



I hereby state that I have reviewed and understood the contents of the above-identified specification, 
including the claims, as amended by any amendment(s) referred to above. I acknowledge the duty to 
disclose all information which is materia! to patentability as defined in 37 CFR 1 .56. 

Foreign Application(s) and/or Claim of Foreign Priority 

i hereby claim foreign priority benefits under Title 35, United States Code Section 1 1 9 of any foreign application(s) for patent or 
inventor(s) certificate listed below and have also identified below any foreign application for patent or inventor(s) certificate having a 
filing date before that of the application on which priority is claimed: 



COUNTRY 


APPLICATION NUMBER 


DATE FILED 


PRIORITY CLAIMED UNDER 35 U.S.C. 1 1 9 








YES: NO: 








YES: NO: 



Provisional Application 

I hereby claim the benefit under Title 35, United States Code Section 1 1 9{e) of any United States provisional application(s) listed 
below: 



APPLICATION SERIAL NUMBER 



FILING DATE 



U.S. Priority Claim 

I hereby claim the benefit under Title 35, United States Code, Section 120 of any United States application(s) listed below and, insofar 
as the subject matter of each of the claims of this application is not disclosed in the prior United States application in the manner 
provided by the first paragraph of Title 35, United States Code Section 1 12, I acknowledge the duty to disclose material information as 
defined in Title 37, Code of Federal Regulations, Section 1 .56(a) which occurred between the filing date of the prior application and the 
national or PCT international filing date of this application: 



APPLICATION SERIAL NUMBER 


FILING DATE 


STATUS {patented/pending/abandoned) 





















POWER OF ATTORNEY: 

As a named inventor, I hereby appoint the following attorney^) and/or agent(s) to prosecute this application and transact all business in 
the Patent and Trademark Office connected therewith: 



Customer Number 022879 



Place Customer 
Number Bar Code 
Label here 



Send Correspondence to: 
HEWLETT-PACKARD COMPANY 
Intellectual Property Administration 
P.O. Box 272400 

Fort Collins, Colorado 80528-9599 



Direct Telephone Calls To: 

Robert Sismilich 
(858) 655-8329 



I hereby declare that all statements made herein of my own knowledge are true and that all statements 
made on information and belief are believed to be true; and further that these statements were made with 
the knowledge that willful false statements and the like so made are punishable by fine or imprisonment, or 
both, under Section 1001 of Title 18 of the United States Code and that such willful false statements may 
jeopardize the validity of the application or any patent issued thereon. 



Full Name of Inventor: Edward J Van Liew 

Residence 
Post 



Citizenship: 



5468 Gooseberry Way Oceanside CA 92057 



US 




"Tnvei 

Rev 11/99 (DecPwr) 



Date 

(Use Page Two For Additional Inventor(s) Signature^)} 



Page 1 of 2 



DECLARATION AND POWER OF ATTORNEY 
FOR PATENT APPLICATION (continued) 



ATTORNEY DOCKET NO. 10990102-1 



Full Name of # 2 joint inventor: Nobuko M Nathan 
Residence: 
Post Office Address: 



Citizenship: JP 



1442 Tzena Way Encinitas CA 92024 



Same as residence 



inventor s signature 



Date 



Full Name of # 3 joint inventor: Birsen Ulupinar 
Residence: 
Post Office Address: 



Citizenship: TR 



inventory sign 



gnature " ~ 



8492 Via Sonoma, #1 La Jolla CA 92037 i7^K7 Gicc^aAqP ^l/%M 
Same as residence SoiA IV^o OA ^917^ 



4/7 



Date 



Full Name of # 4 joint inventor: 

Residence: 

Post Office Address: 



inventor's Signature 



Citizenship: 



Date 



Full Name of # 5 joint inventor: 

Residence: 

Post Office Address: 



inventor s signature 



Citizenship: 



Date 



Full Name of # 6 joint inventor: 

Residence: 

Post Office Address: 



inventor's signature 



Citizenship: 



Date 



Full Name of # 7 joint inventor: 

Residence: 

Post Office Address: 



inventor s signature 



Citizenship: 



Date 



Full Name of # 8 joint inventor: 

Residence: 

Post Office Address: 



Citizenship: 



inventor s signature 



Date 



Rev 11/99 (DecPwr) 



(Use Page Two For Additional Inventor(s) Signature^)) 



Page 2 of 2 



